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1.  Objective 


This  research  is  to  develop  a  self-improving  system  that  is  capable  of  performing  concept 
formation  for  associating  materials  and  process  design  using  inductive  and  deductive 
coupling  techniques. 

2.  Status  of  Effort 

The  first  phase  of  the  work  focuses  on  developing  an  artificial  neural  network  learning 
for  function  approximation.  The  objective  for  neural  network  function  approximation  is 
to  learn  the  input-output  mapping  efficiently.  Different  with  traditional  gradient  descent- 
based  search  algorithm  such  as  back-propagation  and  its  variations,  we  have  developed  a 
rapid  neural  network  unsupervised  learning  algorithm.  Not  only  the  learning  algorithm 
can  update  the  weights  of  the  neural  networks  instantly  for  a  newly  added  input,  but  also 
it  can  update  the  weight  for  a  newly  added  neuron  immediately.  We  have  applied  this 
algorithm  and  architecture  to  different  applications,  such  as  for  the  mapping  and 
prediction  of  an  infrared  laser  data  set,  a  chaotic  time-series,  a  monthly  flour  price  data 
set,  and  identification  of  a  non-linear  system. 

The  second  phase  of  the  work  focuses  on  developing  an  artificial  neural  network  learning 
algorithm  for  time-series  prediction.  A  dynamic  stepwise  updating  algorithm  is  proposed 
to  update  the  weight  of  the  system  in-situ.  We  also  focus  the  work  on  regularization  and 
development  of  functional  space  theory.  It  is  well  known  that  the  goal  of  neural  network 
function  approximation  is  not  to  learn  an  exact  representation  of  the  training  data,  but 
rather  to  build  a  statistical  model  for  smooth  interpolation.  In  practical,  if  a  network  is 
over-fitting  to  the  noise  on  the  training  data,  it  will  give  a  poor  generalization.  To  solve 
this  problem,  there  are  two  main  approaches:  model  selection  and  regularization.  Model 
selection  (the  main  focus  of  the  third  phase)  for  a  feed-forward  neural  network  requires 
choosing  the  number  of  hidden  neurons  and  connection  weights.  The  common  statistical 
approach  for  model  selection  is  to  estimate  the  generalization  error  for  each  model  and  to 
choose  a  model  with  minimum  error.  While  regularization  approach  involves 
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constraining  or  penalizing  the  solution  of  the  estimation  problem  to  improve  network 
generalization  ability  by  smoothing  the  predictions. 

The  third  phase  of  the  work  focuses  on  model  selection.  We  have  successfully  used  both 
neural  networks  approach  and  neuro-fuzzy  approach  to  select  the  most  important  features 
(attributes)  for  function  mapping  and  construct  a  model  with  a  minimum  generalization 
error.  We  have  applied  this  approach,  successfully,  identifying  material  properties  and 
predicting  ternary  systems  compounding  formation. 

3.  Accomplishments/New  Findings 

3.1  Research  Highlights  and  Contribution 

The  first  phase  work  focuses  on  designing  a  new  single-hidden  layer  artificial  neural 
network  architecture  and  an  instant  learning  algorithm  that  rapidly  decides  its  weights. 
Improving  the  learning  speed  of  back-propagation  (BP)  and  increasing  the  generalization 
capability  of  the  networks  have  played  a  center  role  in  neural  network  research.  Apart 
from  multi-layer  network  architectures  and  the  as  the  BP  algorithm,  various  simplified 
architectures  or  different  non-linear  activation  functions  have  been  devised.  Among 
those,  a  so-called  flat  networks  including  functional-link  neural  network  and  radial  basis 
function  network  have  been  proposed.  These  flat  networks  remove  the  drawback  of  a 
long  learning  process  with  the  advantage  of  learning  only  one  set  of  weights.  Most 
importantly,  the  literature  has  reported  satisfactory  generalization  capability  in  function 
approximation. 

We  have  developed  a  one-step  fast  learning  algorithm  and  a  stepwise  update  algorithm 
for  training  the  flat  networks.  The  algorithms  are  developed  based  on  the  formulation  of 
the  network  that  has  a  set  of  linear  system  equations.  The  most  significant  advantage  of 
the  stepwise  approach  is  that  the  weight  connections  of  network  can  be  updated  easily, 
when  a  new  input  is  given  later  after  the  network  has  been  trained.  The  weights  can  be 
updated  easily  based  on  the  original  weights  and  the  new  inputs.  The  stepwise  approach 
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also  is  able  to  update  weights  instantly  when  a  new  neuron  is  added  to  the  existing 
network  if  the  desired  error  criterion  cannot  be  met.  With  this  learning  algorithm,  the 
networks  become  very  attractive  in  terms  of  learning  speed. 

The  second  phase  focuses  on  the  work  of  regularization  and  development  of  functional 
space  theory.  Combing  our  approach  and  unsupervised  learning  algorithm,  we  have 
extended  the  capability  of  discovery  methods  on  materials  research,  such  as  predicting 
material  properties.  Our  regularization  approach  combines  orthogonal  least  square 
method  and  subset  selection.  The  benefit  of  the  research  is  to  reduce  number  of 
regressors  (neurons)  in  the  functional  mapping.  If  the  network  has  too  many  regressors,  it 
will  cause  the  network  to  be  overly  sensitive  to  the  training  data,  which  often  results  in 
poor  generalization.  We  have  developed  an  orthogonal  function  transformation  and  an 
orthogonal  functional  basis  neural  network  (OFBNN)  based  on  this  transformation.  We 
have  applied  this  approach  to  electro-optic  (EO)  data  of  several  semiconductor 
compounds.  The  OFBNN  model  has  been  used  to  learn  a  function-related  attribute  of 
compounds,  which  form  a  “cluster”  based  on  similar  attribute  values.  We  are  able  to  learn 
the  function  and  to  predict  the  feature  value  of  compounds  in  a  cluster.  The  result  of  the 
EO  data  gives  us  a  very  promising  result. 

In  the  third  phase  of  the  work,  the  focus  is  on  the  model  selection.  We  have  applied 
neural  networks  and  neuro-fuzzy  hybrid  model  to  explore  materials  and  process 
discovery.  We  have  successfully  identified  material  properties  that  determine  the  internal 
structure  of  ternary  system  compounding  forming.  Our  neural  network  learning  algorithm 
suggests  that  one  of  the  features,  i.e.,  Mendeleev  number  dominates  more  than  90%  of  the 
accuracy  of  compound  formation.  Our  nemo-fuzzy  approach  for  model  selection  suggests 
that  Zunger  difference.  Energy  difference.  Average,  Variance,  and  Ratio  of  Mendeleev 
number  play  the  most  important  role  for  the  formation  and  for  future  prediction.  This  is 
the  model  with  the  minimum  generalization  error. 
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3.2  Relevance  to  the  Air  Force  Mission  and  Potential  Applications 

The  proposed  function  approximation  approach  can  be  used  in  Pulse  Laser  Deposition 
(PLD)  application.  PLD  is  an  advanced  process  used  to  deposit  a  wide  variety  of  thin 
coating  for  applications  ranging  from  Air  Force  to  civilian,  superconductors  to  solid- 
lubricants.  One  of  components  of  the  PLD  process  modeling  is  focused  on  finding  static 
nonlinear  relationships.  Our  approach  is  perfect  for  this  purpose. 

The  proposed  dynamic  updating  approach  can  be  used  in  Rugate  filter  processing 
application.  Rugate  filter  processing  is  an  advanced  process  used  to  deposit  a  variety  of 
coating  for  applications  for  Air  Force.  The  process  needs  in-situ  monitoring  and  control. 
The  rapid  neural  network  learning  model  can  be  used  as  part  of  an  intelligent  hierarchical 
control  system  to  guide  the  real-time  feedback  control  of  the  process. 

The  regularization  approach  solves  over-fitting  problem  for  the  neural  network  mapping. 
With  regularization,  the  training  is  “immune”  to  the  noise  on  the  training  data,  which 
exist  in  most  process  data  collection.  The  result  gives  us  a  very  satisfactory 
generalization. 

The  proposed  approach  is  a  major  potential  strategy  for  accelerating  materials  design. 
The  goal  is  to  explore  more  efficient  methods  of  organizing  materials  data  using  both 
supervised  and  unsupervised  neural  networks  approach.  Our  approach  can  determine  and 
predict  ternary  system  compounding  forming.  Early,  the  entire  research  group  that 
includes  several  scientists  and  engineers  associated  with  Wright-Patterson  AFB  has 
discovered  that  the  Mendeleev  number  could  be  used  to  predict,  with  99%  of  accuracy, 
the  occurrence  of  a  compound  for  any  multi-element  (i.e.,  binary,  ternary  and  etc.) 
materials  systems.  This  discovery  has  save  time  and  resource  in  exploring  future 
materials  systems.  From  this  preliminary  result,  we  have  approximately  narrowed  down 
the  search  space  of  new  materials.  Furthermore,  we  extend  the  neural  networks  model  to 
neuro-fuzzy  model  selection  such  that  we  can  discovery  most  important  features  to 
predict  the  occurrence  of  a  compound  formation. 
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•  Technical  Committee  of  1 994  Int'l  Conference  on  Neural  Networks  in 
IEEE  World  Congress  on  Computational  Intelligence,  June  26-July  2, 
1994,  Orlando,  FL. 
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6.2  Consultative  and  Advisory  Functions 

Serve  as  a  visiting  research  scientist  with  Materials  Process  Design  Branch, 
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7.  New  Discoveries,  Inventions,  or  Patent  Disclosures 
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Invention  number  21134,  U.S.  Patent  number  5485390,  Jan,  1996,  S.  R. 
LeClair,  Y.  H.  Pao,  T.  E.  Westhoven,  H.  N.  Al-Kamhawi,  C.  L.  Philip 
Chen,  A.  G.  Jackson,  A.  C.  Chemaly. 

•  Orthogonal  Functional  Basis  Method  for  Function  Approximation,  Air 
Force  Invention  No.  D00211,  Dec.  1996,  Y.  Cao,  C.  L.  P.  Chen,  and  S.  R. 
LeClair. 
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•  Dr.  Chen  was  promoted  to  the  rank  of  Associate  Professor  in  the  year  1995. 

•  Research  Excellent  Faculty  Award,  College  of  Engineering  and  Computer 
Science,  Wright  State  University,  1997. 

•  Senior  Research  Fellow,  National  Research  Council,  National  Academy  of 
Sciences/Engineering,  1997 
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